Scientific Data Compression Through Wavelet Transformation

Scientific data gathered from simulation or real measurement usually requires 64 bit floating point numbers to retain accuracy. Unfortunately, these numbers do not lend themselves to traditional compression techniques, such as run-length or entropy encoding used in most general compression schemes. A different approach is to recognize that a reversible signal transformation can encode the information using less information. Perhaps the most useful technique for this domain is the wavelet, which goes beyond traditional Fourier transforms by effectively capturing time and frequency information. The result is that most of the pertinent data is concentrated into a smaller range of values. At this point the data can be thresholded and still retain a high percentage of the “energy” of the signal. Subsequent reconstruction introduces only minor errors depending on the level of thresholding and the wavelet used. Because the data has been thresholded, there exists a large number of zeros which can subsequently be compressed using traditional techniques.